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PART  I INTELLIGENT  EDUCATIONAL  DIALOGUE  SYSTEM  COMPONENTS 


OVERVIEW 

Part  I describes  the  components  of  the  system  in  sufficient  detail  to  motivate 
the  analysis  of  a Socratic  teaching  strategy  given  in  Part  II.  Certain  aspects  of  the 
system’s  structure,  particularly  with  respect  to  issues  of  control  structure  and  inter- 
process communications  are  glossed  ever  here.  These  issues  are  considered  more 
thoroughly  elsewhere  (Neches,  1976);  a brief  summary  is  necessary  to  motivate  this 
section. 

I make  an  effort  to  expedite  studies  of  tutoring  by  providing  a meta-system  in 
which  tutoring  strategies  are  easy  to  specify;  this  provides  a formalism  for  studying 
and  comparing  different  teaching  strategies  in  a common  conceptual  framework, 
tvtodels  of  patficular  teaching  strategies  can  then  be  built  and  studied,  as  Part  II 
demonstrates. 

These  goals  affect  several  facets  of  the  system's  organization.  The  system  is 
divided  ii^to  a number  of  separate  components,  which  might  best  be  thought  of  as 
independent,  very  large,  "Knowledge  Sources".  This  organization  is  nmtivated  primarily 
by  consideraions  of  pedagogy  and  methodology,  rather  than  psychology.  Decomposing 
the  system  into  components  (each  of  which  represents  a major  issue  in  Intelligent  CAI 
and  tutorial  modelling)  simplifies  discussions  of  internal  system  operation*  since  the 
processes  have  been  made  explicit.  The  decomposition  simplifies  presentation  of 
interacting  parallel  processes,  since  each  coraponent  can  be  treated  simply  as  a "black 
box"  wl'ion  not  itself  under  consideration.  Another  virtue  of  this  organization  is  that  it 
encourages  dealing  with  each  of  the  major  issues,  because  a particular  model  must 
further  specify  the  gem*ral  con^ponents  provided  in  order  to  function. 

THE  COMPONENTS  QE  THE  SYSTEM 

The  system  is  viewed  as  having  seven  major  components: 


(1)  an  lu/nii/Onifiui  --  translates  the 

student's  language  into  the  system's  and  vice  versa; 

(2)  7’o/irV  Kiwwtrdff<*  Rrfimtimtntioit  (TKH)  --  holds 
all  information  about  the  subject  to  be  taught; 

(3)  htndrl  of  the  Student  (hfoS)  — holds  student 
information  used  by  processes  such  as  Planner  and 
I/O  Handler; 

(4)  l^tnnnor  --  "tunes"  the  system  to  the  student's 
particula.*  needs,  determining  the  topics  presented, 
thcir  sequence  and  manner  of  presentation,  and 
level  of  detail; 
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(5)  So^Mon  llisioiy  --  records  the  interaction  between 
r.tudent  and  system,  as  well  as  intermediate  internal 
states  of  the  other  component  processes; 

(6)  --  makes  real  world  information 
available  to  the  system; 

(7)  QiK’stion-Answannff  (Q/A)  --  guides  information 
retrieval  from  the  other  components  (e.g.,  TKR)  by 
acting  as  a central  control. 


The  last  three  components  are  included  here  for  the  sake  of  completeness. 
Since  they  are  not  directly  relevant  ^o  the  issues  discussed  in  Part  II.  This  section’s 
scope  will  be  confined  to  the  first  four  components. 

THE  INPUT/OUTPUT  HANDLER 

The  I/O  Hiindlt?r’s  first  duty  is  receiving  communications  from  the  student, 
deternnning  their  meaning/intent,  and  representing  them  in  an  internal  language.  This 
task  uses  the  Model  of  the  Student,  Pragmatics,  and  Topic  Knowledge  Representation 
(to  obtain  information  about  vocabulary,  plausibility  of  its  interpretations,  etc.),  as  well 
as  Session  History  (to  interpret  input  with  respect  to  the  current  context). 

The  I/O  Hiindler's  second'duty  is  generating  output  to  the  student  based  on  an 
interna!  r(?quest  generated  by  one  or  more  of  the  other  processes.  The  request 
describes  the  kind  of  information  to  transmit  (e.g.,  whether  or  not  an  answer  was 
corriict)  and  controls  variables  such  as  level  of  sophistication,  but  is  not  itself  an 
English-level  statement.  Thus,  no  other  process  is  concerned  with  language 
processing,  ail  hough  the  I/O  Handler  calls  on  Planner,  Pragmatics,  Topic  Knowledge 
R ‘pi  ('sentation,  and  Model  of  the  Studei>t  to  obtain  ll^formation  used  in  constructing 
the  actiuil  lext. 

Tl'.ough  much  more'  general,  this  organization  is  similar  in  many  respects  to  the 
ISI  Infonnalion  Avitoir,.dion  Tutor  (Rothc*nberg,  197‘j),  which  has  pre-storod  ii^formation 
varying  along  dimensions  of  "verbosity"  and  "sophistication"  Messages  arc 
constructed  for  a particular  student  by  using  a "User  Profile"  to  select  locations  along 
the  two  dimensions. 

TOPIC  KNOV^LEDGE  REPRESENTATION 

The  system  was  designed  to  teach  two  sorting  algorithms  from  computer 
science:  the  "straight  selection  sort"  and  the  "binary  tree  sort"  (Knuth,  1973,  pp.  139- 
I'U  '328). 

In  tutormi  protocols,  four  kinds  of  Knowledge  about  algorithms  can  be 
distii^guished.  (The  examples  come  from  actual  tutorials.  They  are  presented  in 
boxes.)  The  four  kinds  are: 


-2- 


COMPONENTS 


(1)  Sirnrturn  Iho  algorithm  viewed  in  terms  of  its 
sub-algorithms  and  the  flow  of  control  between 
them.  Box  1 shows  a simple  example.  Structure 
knowledge  is  mulli-levelcd,  as  Box  2 Illustrates. 

(2)  Mniirntions  - reasons  both  for  the  purpose  and 
the  design  of  algorithms.  This  subsumes  real  world 
examples  (Box  3),  interactions  between  parts  (Box 
^),  and  weighing  of  advantages  (Box  5). 

(3)  l^rofioriira  the  basic  properties,  advantages,  and 
disadvantages  of  algorithms;  Box  6 is  an  example. 

(^>  Effort f.  i.e.,  "what  happens  If...".  Box  7 shows  a 
tutor  explaining  a proposal’s  effects. 


Insert  Boxes  1-7  about  here 


How  can  these  different  Kinds  of  Knowledge  be  represented?  The  answer 
proposed  here  toiribinos  procedural  and  declarative  reprcst-ntatioiv.. 

An  algorithm’s  formal  structure  i'--  expressible  in  a sirurtnro  diaaramt  a 
formalism  for  representing  both  organization  and  flow  of  control  in  computer  programs. 
The  underlying  principles  are  those  of  structured  programmii^g  (Dahl,  Djikstra,  S:  Hoare, 
1972).  Structure  diagrams  represent  programs  as  a tree  structure;  depth  in  the  tree- 
indicates  decomposition  into  simpler  functions,  left-right  order  of  branches  indicates 
sequence  of  processing. 

Figure  I shows  the  "build  tree"  algorithm's  diagram  ^ encoded  in  a "sirurturo 
nr/wnrlf,  which  uses  a modified  version  of  the  LNR  memory  representation  (Norman, 
Rumelliart,  and  the  l.NR  Research  Group,  1975).  Although  nodes  usually  consist  only  of 
a name?  and  pointers  to  othetr  nodes,  the  modifications  add  “xirurtttrr  iindrt'  (denoted  in 
figures  by  double*  lines)  having  a broader  range  of  associated  information;  a name,  a 
type  (choice,  action,  or  repetition),  and  a description  of  its  contents.  Structure  nodes 
represent  either  "tasks',  processes  of  unspecified  method  (e.g.,  sorting),  or 
"ntaonthms",  processes  of  specified  method  (e.g.,  binary  tree  sorting).  It  is  sometimes 
useful  to  think  of  algorithms  as  tasks  for  which  only  one  method  is  specified. 


insert  Figure  I about  here 


Task  structure  nodes  have  a "MCTIOD"  relation  to  a token  of  the  n-ary 
predicate  "OR",  which  spcrcifics  alleinative  algorithms  for  accomplishing  the  task. 
Algorithms  have  a *W"  ImK  to  either  a structure  node  or  a token  of  the  n-ary 
predicate  "00  IN-ORDER",  which  indicates  the  sequence  m which  components  of  the 
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alpoi  ithm  arc  carrind  Out.  The  DO- IN-ORDER  and  OR  predicates  are  iilustrated  in 

Fip,ur(‘  2. 


Insert  Figure  2 about  here 


Figure  2 also  illustrates  the  "HAS-PROP"  predicate,  whicn  asserts  that  some 
subject  S HAS-PROPerty  P.  The  token  in  Figure  2 asserts  that  the  binary  tre*'  sort 
HAS-F’ROP  that  its  speed  is  "fast”.  Use  of  the  HAS-PROP  predicate  oiiows  assertions 
about  properties  to  become  arguments  of  other  predicates,  while  still  allowing  the 
effect  of  maintaining  a property  list  (since  a structure  nooe  can  be  searched  for  all 
associated  HAS-PROP  tokens). 

Motivalions  will  be  represented  propositionally  m a modified  story  grammar 
(Rumclliarl,  1975),  smiplified  by  using  assumptions  sucii  as  that  an  algorithm  is  selected 
either  because  its  properties  mr^et  a goal’s  requirement  or  because  its  alternatives 
don  l.  This  is  illustrated  by  the  "WHEN",  "REQUIRF",  and  "USE"  predicates  of  Figure  2. 
These  state  the  motivation,  "When  sorting  requires  speed,  use  the  binary  sort  because 
it’s  fast." 


Process  Knowledge  requires  the  ability  to  simulate  and  examine  the  execution  of 
an  algorithm.  Scragg’s  thesis  on  answering  process  questions  (Scragg,  i975)  suggests 
an  attractive  approach,  representing  processes  as  procedures.  Questions  are 
answernd  by  examining  the  procedures'  code,  and/or  executing  the  code  under  special 
supervision.  The  structure  network  representation  can  provide  all  of  the  information 
needed  in  Scragg's  approach. 

The  representational  scheme  just  described  captures  the  four  kinds  of 
Knowledge  about  algO'-ithms  to  at  least  a first  degree  of  approximation.  It  is  fairly 
easy  to  see  why  tins  knowledge  is  useful  to  both  tutor  and  student.  The  different 
knc)W'lcdge  types  make  for  an  extremely  rich  node  network,  in  which  concepts  are 
firmly  anchoiod  by  many  links  to  nodcjs  for  other  concepts.  The  latter  three 
Knowledge  ty(x?s,  constituting  Knowledge  an  algorithm  as  opposed  to  knowledge 

of  it,  allow  .1  student  who  has  forgotten  the  structure  to  recompute  it.  That  is. 
Knowledge  from  any  three  of  the  types  often  provides  sufficient  constraints  on  the 
situation  to  purmit  infcroncmg  knowledge  of  the  fourth  typo  in  relative  safety. 

These  argumr^nts  for  the  knowledge  types’  utility  are  best  understood  in  the 
{ontc'xt  of  Noi  mi.iiFs  *wi'b  Icai  ning*’ concept  (Norman,  1973,  197^).  Norman  argues  that 
a of  nth  ii^ter(  onnt?ctions  is  essential  for  effective  learning  and  retention  of 

ttie  Kir^d  of  concepts  wc  wish  to  teach.  Tt»c  teaching  process,  he  believes,  must  consist 
of  fn.my  pass»,'s  through  a network  of  Knowledge  to  be  taught.  This  permits  a concept 
to  be  touched  on  many  times,  in  many  different  contexts.  Thus,  a student  benefits 
from  multiple  exposures  to  a concep\  as  well  as  strong  anchonr'ig  to  other,  already 
famili.ii  concepts  Si»>cc  the  knowlodgo  leprcscntation  presented  here  admits  both 
tuerarrhic jl  arxl  non-hierar cnical  relationships  between  concepts,  it  allows  the 
f apiiliility  to  lake  these  multiple  passes  through  the  Knowledge  network  at  varying 
levels  of  ciet.)il  and  sophistication.  This  is  a key  to  the  web  learning  approach;  unlike 
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the  contracting  approach  of  "sequential  learning",  web  learning  ensures  that  the 
student  always  maintains  a coherent  knowledge  structure.  The  structure  at  times  may 
bo  incomplete,  but  it  will  never  be  unstable. 

MODEL  OF  Tl€  STUDENT  (MoS) 

Several  examples  in  this  paper  illustrate  a responsive  system’s  need  for  a 
database  of  information  about  the  student’s  current  knowledge  state  and  general 
propensities. 

Direct  and  inferential  data  about  students  is  gathered  by  examining  their  input 
via  the  I/O  Handler,  Topic  Knowledge  Representation,  and  Session  History.  In  addition, 
analyses  of  its  data  performed  elsewhere  can  feed  back  to  MoS.  The  Model  of  the 
Student  must  supply  its  information  to  several  other  processes,  working  particularly 
closely  with  the  I/O  Handler  and  Planner.  The  I/O  Handler  uses  the  information  both  in 
understanding  student  input  and  in  generating  system  output.  Planner  uses  the 
information  in  selecting  teaching  rules. 

Information  must  be  collected  when  available  rather  than  when  requested, 
recjijifing  the  ability  to  anticipate  what  information  will  be  needed.  This  ability  is 
cir.bodieci  in  processes  called  Knowlrdf(c  Stntr  Conriitiont  (KSCs).  These  can  bo 
thought  of  as  procedures  processed  partly  by  the  Planner  and  partly  by  MoS.  The 
definition  of  a Knowledge  State  Condition  contains: 


(1)  an  hi/onnotion  roihrter  — instructions  specifying 
what  information  is  needed  and  how  it’s  to  be 
obtained; 

(2)  an  nnniyxhi  ti*rfion  --  an  algorithm  for  determining 
how  well  the  condition  has  been  met  by  examining 
the  information  collector’s  data; 

(3)  a --  an  algorithm  for  computing  the 

condition’s  importance  as  a function  of  the  system’s 
current  stale. 


If^  this  •..cheiite,  Planner  processes  the  weight. r\g  information,  MoS  the  information 
collcctois.  Tfie  analysis  section  is  si'iarcd  (Planner  invokes,  SteS  processes).  Each 
seciton  of  a KSC  runs  separately.  Information  collectors  function  like  the  demons  of 
Scdfntigc  and  Ntussei  (1960),  invoked  at  any  time  by  student  input  or  internal  system 
slal.s.  Analysis  and  wcightn^g  arc  top-down  functions,  invoked  only  by  direct  call. 

plannir 

Planner  is  responsible  for  choosing  and  presenting  topics.  Students  may  also 
introduce  topics;  Planner  must  notice  and  respor\d  appropriately  when  this  occurs. 
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The  of  Planner  ii.  a set  of  piittern-action  rules  which  will  be  referred  to  in 

till*.  papc*r  ai.  "tvarhinf:  ni/f.s”.  A teaching  strategy  can  be  considered  to  be  a set  of 
related  teaching  rules.  Part  two  of  this  paper  will  present  a set  of  teaching  rules  for 
the  Socratic  teaching  strategy. 

Planner  perfornis  its  role  primarily  by  searching  for,  and  then  applying,  an 
appropriate  teaching  rule.  This  rule-selection  process  starts  with  a series  of  policy 
decisions  about  various  aspects  of  the  presentation.  The  policy  decisions  serve  both 
to  reduce  the  set  of  teaching  rules  which  might  apply,  and  to  constrain  the  application 
of  the  rule  finally  selected. 

Teaching  rules  consist  of  a condition-part  and  an  action-part,  where  condition- 
parts  describe  situations  for  which  the  action-part  is  well-suited.  Condition-parts  are 
represented  as  the  Knowledge  State  Conditions  described  in  the  discussion  of  Model  of 
the  Student.  The  set  of  teaching  rules  applicable  at  a given  point  in  time  consists  of 
those  rules  whose  conditions  are  satisfied  in  the  current  context. 

There  oi  e three  stages  of  decision-making  in  rule-selection;  policy-smihg, 
and  conf Un-resolution.  (Condition-testing,  already  discussed,  is 
simply  the  process  of  determining  which  rules  are  applicable  in  the  current  context.) 

Policy-setting  is  the  process  of  making  decisions  about  various  aspects  of  the 
presentation.  In  examining  tutorial  protocols,  four  critical  aspects  (or  dimensions)  of 
the  presentation  appear: 


(1)  level  --  the  degree  of  sophistication; 

(2)  sequence  — the  Order  in  which  concepts  are 
presented; 

(3)  momirr  — e g.,  question-onented  or  expository; 

(A)  initiniitie  -■  who  initiates  exchanges  (tutor, 
student,  or  both). 


For  the  remainder  of  this  paper,  primarily  the  first  dimension,  level  of 
presentation,  will  be  considered.  For  all  dimensions,  it  will  be  assumed  that  general 
policy  decisions  (i.e.,  choices  of  teaching  strategy)  arc  fixed;  only  specific  decisions  will 
be  described.  ^ 


CorT'ict’fcsolution  is  the  process  of  selecting  a teaching  rule  to  apply  from  the 
set  of  alternatives  (those  rules  with  satisfied  conditions).  Planr^er  uses  several 
heuristics  m this  process: 


(i)  l*olir\:  Reject  alternatives  inconsistent  with  the 
current  policy  decisions. 
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(2)  Coniru:  Offau*  the  “context"  to  be  the  current 
dependent  v»ii  i.ibic  and  immediate  factorc. 

Then,  if  conditions  for  two  different  rules  are  met, 
but  the  Situation  satisfying  the  conditions  for  one  is 
more  ctoscly  related  to  the  context,  choose  that 
rule. 

(3)  Snrrifiriiy:  If  conditions  for  two  different  rules 
are  met,  prefer  the  rule  with  more  specific 
conditions. 

Artion:  f?o)ect  any  rules  with  action-parts 
that  cannot  bo  carried  out. 


Ttiere  is  also  a houristic  for  conflicts  of  a rule  with  itself: 


/h7»rtu/«»rif  /'iiuir  rend  rut  Situnlions:  If  two 
separate  situations  satisfy  conditions  for  the  same 
strategy,  and  one  situation  is  dependent  on  the 
other,  apply  the  rule  to  the  inciependent  situation. 


In  the  sample  dialogue  analysis  following,  the  claim  will  be  made  that  the 
Planner's  policy  decisions,  in  conjunction  with  its  conflict-resolution  heuristics,  impose 
sufficient  constraints  to  determine  a unique  rule  among  alternatives. 

This  control  ' true turn  is  similar  in  many  respects  to  a production  system  (Newell. 
1073;  Newrii  and  K4<Dermott,  197h)  A production  system  consists  of  a set  of  (fata 
t'li'n.r  nt‘.  U alU'd  "Shor  1 rrirn  KVmOry"),  a set  of  “productions",  ancf  a supervisor  whici^ 
haiullr-.  "(Onflict  resolution"  Productions  ate  contingent  instructions  consisting  of 
(onciiiions  p,iiri.*d  with  actions  The  t>asic  cycle  Of  a production  system  has  two  steps. 
In  the  first,  those  productions  with  conditions  satisfied  by  the  configuration  of 
c'leiitc. i>ts  in  Short  Tt‘na  M.‘rriOry  arc  identifrcct.  If  several  are  found  they  are  called  a 
“cOMflut  v»»t“,  lonflut  resolution  means  choorang  a production  from  this  set  according 
to  ‘.Ofitc  pre  -.pecified  criteria.  In  the  second  step,  the  action  part  of  the  selected 
production  IS  i.^xeciited  Tfte  cycle  repeals  uidil  the  first  step  fails  to  firsd  at  least  0>se 
produc  tion. 
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PART  II  --  A FLEXIBLE  SOCRATIC  TEACHING  STRATEGY 


OVERVIEW 

Thio  cliscur.sion  w«!l  focus  m gioatcr  detail  oi^  three  parts  of  the  system 
disc ut. '.eel  m Part  I;  the  I’lanner,  the  Topic  Knowtedge  Representation  (TKR),  and  the 
Model  of  the  Student  (MoS).  The  goal  is  to  demonstrate  how  a particular  teaching 
strategy,  the  Socratic  method,  could  be  implemented  m the  hypothetical  dialogue 

systerr.. 


The  analogy  between  Planner  and  a production  system  becomes  important 
because  of  the  worK  on  anatyi'ing  the  Socratic  method  currently  being  done  by  Allan 
Collins  of  Bolt,  BeraneK,  and  Ni'wman,  Ir\c.  (Collins,  1977). 

Collin;.  i‘.  interested  in  representing  trie  Socratic  method  as  a set  of  very  high 
lev(  1 productions,  each  of  which  suggests  actions  for  the  tutor  to  take  when  some 
Situation  occurs.  His  paper  presents  23  production  rules  for  Socratic  tutoring. 

This  section  <ii*;cusses  Collm;.'  analyse,  and  shows  how  his  productions  could  l)c 
:riij»!err.ented  m an  L'ciuc alional  Dialogue  Systera.  The  systc*m  .s  then  demonstrated  by 
tracing  through  a sample  di.dogue.  In  the  process.  Planner  and  Topic  Knowledge 
Repr i“.t‘ntation  will  lx‘  worl.ed  out  in  detail,  and  the  Model  of  the  Student  wll  be 
iliust'  fited  irdormally . 

Collin;,  points  out  that  for  his  productions  to  constitute  a complete  specification 
of  tile  Socratic  method  requires  a higher  level  theory  of  teaching  strategy  be  added 
on,  in  order  to  determine  which  rules  are  most  appropriate  to  invoke  in  different 
situat.ons  Although  much  remains  to  be  worked  Out,  this  paper  represents  the 
bfgirxungs  of  suc.h  a theory.  Of  particular  importance  are  the  division  of  strategies 
into  multiple  aspects  and  ihe  conftict'resoiution  heuristics,  both  presented  in  the 
prececJ.ng  Planner  discussion  and  furltier  illustrated  later  in  this  section. 

SUMMARY  or  COLLINS'  ANALYSIS 

The  Key  idea  m the  Socratic  method,  according  to  Collms,  it  that  o?  “causal 
ctcpenclencicr**.  the  ur'iderlymg  rciationships  between  facts.  The  method  forces 
students  to  learn  to  tK*nk  for  themselves,  to  induce  general  pniKiples  from  knowledge 
of  specdic  cases,  and  to  apply  those  general  principles  to  new  cases.  Thus,  throe 
types  of  tilings  are  being  taught:  1)  specific  information;  2)  causal  (^pendencies:  and 
3)  reasoning  skills  --  mainly  prrt«»ining  to  hypothesis  generation  and  testing. 

A fattly  Mm. pie  reprrsenl.'it.ort  of  the  knowledge  to  br  taught  is  consiciered 
sati‘.(ac!ory  lew  the  pui poses  of  his  tiiscus'>ion.  situations  related  to  each  other  by  one 
bring  either  necessary  or  sulficicnt  for  the  other.  The  simplicity  is  maefe  possible  by 
pi.ic.ng  pnmai  y emphasis  On  causai  dependencies  (which  are  manageable  m such  a 
framework;,  ai\d  On  reasoning  smiIs  (which  are  taught  implicitly  by  the  production 
rule-.,  *#nd  thw.  need  not  be  represented  formally). 

Beloie  conMtk*r»f‘.g  Ihe  content  of  CoUms’  production  rules,  il*s  necessary  to 
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under '..1  <incl  I ho  torrrnnolof;y  ur.od  lo  r.t-.te  them.  An  explanation  ic  fnoct  easily  made  in 
the  context  of  an  example. 

Imagine  that  a tutor  was  trying  to  teach  a student  about  a mathematical  function: 

x^  if  X > 0 
f(x)  y 

^ -x*^  otherwise. 

Then  "f(x)"  is  the  dependent  variable. 

There  are  two  sufficient  factors  determining  the  dependent  variable  *'f(xr.  They 
are  "x*^"  and  "-x^".  These  are  sometimes  called  or-connected  factors  because  one  or 
the  other  is  sufficient.  Collins  would  diagram  the  relationship  as  in  Figure  3a. 


Insert  Figure  3 about  here 


It  is  also  possible  lo  have  necessary  factors.  ”X^‘‘  and  ‘’-x^‘*  each  have  one 
necessary  factor  --  “x^"  requires  "x  ^ 0",  and  ’’-x*^"  requires  "x  < 0".  This  would  be 
diagrammed  as  it'»  Figure  3b. 

If  there  were  m;re  than  one  necessary  factor,  they  would  be  diagrammed 
similarly  lo  or-connected  factors  except  that  the  arrows  would  be  labeled  with  “oiirf", 
and  would  be  called  "a»'d-connected". 

Several  terms  describe  relationships  between  factors  and  the  dependent 
variable.  Any  path  connected  by  arrows  is  a chain.  For  example,  "x  > 0'*,  "x^",  and 
"f(x)",  form  a chain.  A factor  between  two  nodes  on  a chain  is  called  an  intermediate 
factor:  "x^"  is  an  intei  mediate  factor  of  "x  > 0"  and  ”i(x)".  A factor  on  a chain,  but  not 
an  immediate  predecessor  of  the  dependent  variable  is  called  a prior  factor.  "X^"  is  an 
immediate  predecessor  of  "f(x)";  "x  > 0”  is  a prior  factor. 

P<u  1 1<  ular  c .mc;  of  tlie  dependent  variable  is  a call  to  consider  the  factors 
when  some  specific  irdormation  is  given.  For  "ffx)”,  particular  cases  consist  of 
specified  values  of  >:.  Possible  cases  for  f(x)  are  the  set  of  real  numbers:  f(4),  f(-3), 
and  f(/?.7)  are  all  parliculnr  cases. 

I he  values  of  a d(?pcndent  variable  arc  the  results  of  consuJenng  particular 
cases.  The  value  of  "f(x)"  is  16  when  x»^,  -9  when  x«-3,  and  7.29  when  x-2.7. 

f?e levant  factory,  depend  on  particular  cases.  For  example,  "x^'*  is  relevant  when 
X ->  0,  but  not  when  x is  r^cgativc.  Factors  are  either  primary  or  secondary,  depending 
on  whether  they  are  frequently  relevant. 

Teachifvi  ri^»s  are  productions  specifying  actions  to  be  carried  out  when  their 
assoc i.itcd  conditions  are  met.  Collins’  23  Socratic  teaching  rules  are  listed  in  Table  1. 
Consider  a sample  dialogue  about  "f<x)"  to  illustrate  the  first  three  teaching  rules: 


-9- 


SOCRATIC  TEACHING 


Insert  Table  1 about  here 


Tutor:  What  would  f(x)  be  when  x = ^?  {Huh.  J is 
selected  silica  this  starts  a sassioiu  Tho 
nnrlir.n.lar  ca.*ia  of  x = 4 is  saheted  for  tha 
drpaudant  vnrifilila  f(x)^  and  lha  student  is 
asked  to  predict  its  value.) 

Student:  Sixteen. 

Tutor:  Wl^y  16?  (Hula  2 is  .^elected  heenuse  the 

student  has  asserted  a value  for  lha  particular 
en.se.) 

Student:  Because  A is  t;reater  than  zero. 

Tutor:  W)^y  is  that  important? 


("\  > 0"  is  a prior  factor  of  tha  dependent  variahlc 

‘‘f(vf\  This  satisfies  tha  conditions  for  Rule  3.) 

Student:  Because  when  x is  greater  than  zero,  f(x)  is 
x^,  and  tho  square  of  4 is  16. 


Finally,  define  a teaching  strategy  to  be  any  coherent  set  of  teaching  rules. 
TOPIC  KNOV^LEDGE  REPRESENTATION  ISSUES 

The  problem  now  is  mapping  between  the  structures  in  Topic  Knowledge  and  the 
entities  called  for  in  the  conditions  of  Collins*  rules.  Tables  2 and  3 display  this 
mapping.  Note  that  the  condition  entities  are  described  as  items  to  be  computed 
dynamically  with  respect  to  the  current  dependent  variable,  which  may  be  either  a 
task  or  an  algorithm 


Insert  Table  2 about  here 


Table  2 specifies  what  instantiations  of  the  condition  entities  arc  possible  for 
tasks,  along  with  their  identifying  features  in  Topic  Knowledge  Representation.  There 
are  two  groups  of  entities  which  are  associated  with  each  other  (e.g.,  particular  cases 
involving  input  are  always  associated  with  values  involving  outputs).  Table  3 specifies 
the  same  kind  of  information  for  algorithms.  In  this  case  there  are  two  groupings 
again;  the  groupings  differ  in  having  different  types  of  “values**  (outputs  vs. 
properties). 
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Insert  Table  3 about  here 


A SAMPLE  SOCRATIC  DIALOGUE 

l.et  us  illustrate  the  ideas  presented  here  by  applying  them  in  a tutorial  protocol 
analysis  of  a hypothetical  dialogue  between  a tutor  (T)  and  a student  ($>  about  the 
"binary  tree"  sorting  algorithm.  ^ The  dialog  was  constructed  to  maximize  the  number 
of  teaching  rules  and  conflict-resolution  heuristics  illustrated,  but  is  based  on 
transcripts  of  a number  of  real  tutorials  conducted  by  the  author.  ^ The  discussion 
concentrates  on  the  algorithm’s  second  half,  the  recursive  traversal  rule  for  selecting 
items  from  an  already *built  tree.  The  tutor’s  goal  is  to  help  the  student  understand 
the  rule  and  express  it  in  a computer  program. 

It  is  important  to  note  the  assumptions  implicitly  made  about  student  and 
subject.  When  is  the  Socratic  method  useful?  A number  of  conditions  seem  relevant. 
Topics  are  primarily  concerned  with  relationships  between  concepts  (Collins*  "causal 
dopCMidencies").  Students  are  already  familiar  wilh  prerequisite  concepts  (though 
pc'iTiaps  not  their  inter  -relationships).  The  student  is  not  driven  by  a goal  to  acquire 
a particular  Knowledge  unit  (i.e.,  is  content  to  be  led  by  the  tutor).  Also,  the  student 
has  a problem  -solving  attitude  towards  questions  (making  comments  such  as,  "Don’t  tell 
me,  I want  to  figure  it  out  myself"). 


Insert  Dialogue  Transcript  about  here 


Insert  Figures  4 and  5 about  here 


Insert  Boxes  8-12  about  here 


AN  ANAl  YSIS  01  Till  TUTORIAL  DIALOGUE 

Uy  (onsidonnj',  seiccicd  porlionr.  of  the  preceding  dialogue,  this  section  seeks  to 
illut.trate  three  issues; 


(1)  rules  used  by  the  tutor  and  the  method  of  their 
selection  (Planner); 

(2)  Know-lodge  required  about  the  subject  matter,  and 
its  representation  (Topic  Knowledge 
Representation); 
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(3)  knowledge  required  about  the  student’s  knowledge 
state  (Model  of  the  Student). 


The  student’s  programs  and  verbal  statements  will  be  viewed  as  hyp^the  ,es 
about  necessary  or  sufficient  steps  in  carrying  out  a particular  task.  The  purp'.e  of 
the  analysis  is  to  test  the  extent  to  which,  by  utilizing  this  view,  we  can  expl  the 
behavior  of  a flexible,  intelligent  tutor  within  a framework  of  systematic  vie 
applications  to  a specified  data  structure.  ^ Thus,  the  interaction  betw'.?en  e 
student  (S)  and  the  tutor  (T)  will  be  examined  from  the  tutor’s  viewpoint.  On  S’s  turns, 
the  analysis  is  concerned  primarily  with  what  information  can  be  added  to  the  Model  of 
the  Student  at  the  end  of  the  turn.  For  T’s  turns,  the  focus  is  on  the  processes 
determining  the  tutor’s  actions  on  that  turn. 


Line  1 


Planner: 

The  tutorial  starts  at  an  intermediate  level;  it  assumes  that  the  student  is  the 
type  for  which  the  Socratic  method  is  appropriate  (see  above). 

Rule  1 ("Known  case")  is  selected,  since  this  is  the  start  of  the  session.  There 
are  two  possible  actions  (see  Table  1,  Rule  1).  The  first  gives  away  more  information 
and  thus  is  somewhat  easier;  at  intermediate  levels,  the  easier  option  is  taken.  Speed, 
a motivational  factor  in  picking  an  algorithm,  is  selected  as  the  question's  topic. 

Topic  Ktiotvlcd ge  liepreaentntion: 

There  arc  two  sorting  algorithms:  "binary  tree"  and  "straight  selectio**".  The 
binary  tree  sort  is  fast;  when  a quick  sorting  algorithm  is  required,  it  should  be  used 
because  it  has  the  desired  property.  (This  information  is  represented  in  Figure  2.) 

Line  2 

Topic  Ktiotvlcd gc  Kcprcsviitnuoti: 

The  binary  tree  sort  consists  of  building  a tree  and  traversing  it  (illustrated  in 
Figure  2). 

Model  of  the  Student: 

S knows  of  the  tree  building  algorithm,  part  of  only  one  known  algorithm  (which, 
also,  nas  the  property  just  queried).  This  indicaf's  S is  at  least  partly  aware  of  the 
tree  sort. 
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Lino  ^ 


Model  of  ihf.  Stndoul: 

The  student  has  identified  the  two  necessary  factors  for  the  binary  tree  sort 
alcorithm.  This  indicates  that  S knows  the  answer  to  the  query  of  Line  1 (without 
implying  that  the  algorithm  is  completely  understood). 


Lino  5 


Planner: 

The  intermediate  level  and  failure  to  completely  specify  the  algorithm  at  Line  2 
suggest  that  S doesn't  completely  understand  the  algorithm,  leading  to  a sequence 
policy  decision  establishing  the  binary  tree  sort  algorithm  as  the  new  dependent 
variable. 

"Prior  factors"  (fable  1,  Rule  4)  applies  because  "traversing  the  tree"  Is  an 
Immediate  predecessor  factor  of  the  binary  tree  algorithm  (the  dependent  variable). 
"Specify  functional  r(»lationship  (Table  1,  Rule  11)  is  also  a contender  --  S hasn’t 
specified  why  the  binary  tree  sort  is  fast  — but  it  violates  the  "context”  conflict- 
r(?s.c)lution  heuristic  bocaiise  it  applies  to  the  old  dependent  variable. 

Tof)ic  KnowUd ge  Ixrijresentaiion: 

The  factors  of  "traversing  tlie  tree"  are  illustrated  under  the  "traverse"  node  in 
the  shaded  pari  of  Figure  6.  Traversal  Involves  following  the  tree,  starting  with  the 
root.  "Following"  consists  of  checking  that  the  node  is  empty,  and  (if  not)  following  its 
left  link,  lhcr>  printing  it,  then  following  its  right  link. 


Insert  Figure  6 about  here 


Lines  6-7 

Model  of  the  Student: 

The  student  is  unable  to  hypothesize  the  factors  relevant  to  traversing  the  tree. 


Linf^_Fi 

Planner: 

I he*  li*\'<  I of  presentation  policy,  set  at  Line  3,  and  the  choice  of  topic  policy,  sot 
«it  I me*  F),  are  reinforced  by  the  Model  of  the  Student’s  idc^ntification  of  a large 
Knowledge  gap. 
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f3olh  "Av.k  for  relevant  factors”  and  ‘‘Consider  relevant  factors”  (Table  1,  Rules 
17  and  ?3  respectively)  are  candidates,  since  S cannot  make  a prediction.  The  latter 
sufji’est  factors  while  the  former  assumes  the  student  already  knows  them}  the  level 
policy  thus  mandates  Rule  23. 

'i'opic  Knowledge  Ke presentation: 

A property  of  the  tree  built  in  the  binary  sort’s  first  part  determines  the 
traversal  method,  (for  nov*!,  simply  call  this  properly,  "TREE-PROPERTY”.)  This 
connection  is  pro  dded  by  a predicate  chain  linking  “build  tree"  with  TREE-PROPERTY 
and  another  chain  from  TREE-PROPERTY  to  “traverse  tree"  (the  shaded  part  of  Figure 
7).  Respectively,  the  two  chains  assert  that:  1)  "Build  tree”  has  the  property  that  its 
outptu  has  TREE-PROPERTY;  and  2)  ‘Traverse  tree”  has  the  property  that  its  input  has 
TREE-PROPERTY. 


Insert  Figure  7 about  here 


iinos  12-21 

I'opic  Knowledge  lie  presentation: 

Figure  1 shows  the  structure  network  for  the  tree  building  algorithm. 

Model  of  the  Student: 

Comparing  Topic  Knowledge  Representation  (Figure  1)  to  the  student’s  statement 
si'.ows  that  S has  listed  all  factors  of  tree  building. 


Linf>  22 

Planner: 

"intermediate  factors”  ( f able  1,  Rule  3)  is  che  only  applicable  rule.  A motivating 
factor  in  the  algorithm  is  that  the  properties  Of  "build  tree’s”  output  facilitate  the 

travc'is.il  algorithm.  What  is  needed  from  S now  is  the  intermediate  step  between 

‘‘build  tiee"  and  "traverse  tree"  (that  is,  the  facilitating  TREE -PROPERTY). 

’I'opic  Knowledge  lie  presentation: 

The  critical  TREE-PROPERTY  summarizes  as: 

For  any  node  X, 

for  all  nodes  Y m the  tree  with  root  X, 
either:  1)  X is  the  same  as  Y, 
or  2)  Y IS  less  than  X and  is  in  X’s  left  sub-tree, 

or  3)  Y is  greater  than  X and  is  in  X’s  right  sub-tree. 
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The  rcprosentiition  for  TREE-PROPERTY  is  shown  in  the  unshaded  portion  of 
Figure  7 (shaded  rcfgions  represent  connections  discussed  earlier). 


Lines 

Model  of  the  SttLdciit: 

S is  av/aro  of  TREE-PROPERTY  and  that  it  is  shared  by  "build  tree"  and  "traverse 
tree"  (see  Lines  1-4  of  the  dialogue  for  further  evidence). 

I ino  ^r> 

l^lanner: 

Lines  12-21  and  23-24  indicate  that  S knows  some  fairly  detailed  info-'mation, 
wLiich  raises  the  level  of  presentation. 

"Specify  funciiOnal  ndationship"  (Table  1,  Rule  11)  is  the  only  possibility.  TREE- 
f^RQf-’LR'l  Y,  a property  of  the  "traverse  tree"  algorithm  (see  Figure  7,  shaded  region),  is 
al'.r^  a mr>hvation  for  its  steps  (introduced  in  Figure  6,  shaded  region).  Given  a factor 
for  choosing  a value?  (in  this  case,  the  algorithm’s  steps),  Rule  1 1 asks  S to  state  how 
the  factor  determines  the  value.  (Note  that  a value  can  be  a group;  as  here  the 
question  pertains  to  the  ^ of  steps  composing  the  "traverse  tree"  algorithm.) 

TofiU  Ktmtvltdge  Kepietaoitatioti: 

The  "tree  property"  (Figure  7,  unshaded  region)  motivates  the  traversal 
algonlhni  through  Knowledge  about  other  properties/effects  of  traversal  (Figure  6, 
unshaded  r(?gion).  Following  left  links  before  printing  a node  causes  everything  in  its 
left  sub-tree  to  be  printed  b(*fore  it.  Printing  the  node  before  fohowmg  its  right  links 
cnii'.es  every thir.g  in  its  right  sub-tree  to  be  printed  after  it.  Via  a chain  of  HA5- 
PROPs,  "Iraversal"  irdiei  its  the  properly  of  causing  these  two  situations.  This,  together 
witli  TREE-PROPERTY  of  Figure  7,  causes  "iraversal"  to  have  its  output’s  properly  of 
ascending  C'rder. 

That  this  causal  relationship  exists  is  a reaton  to  use  the  set  of  actions  shown 
fc>r  ‘’{raversal"  in  Figuii?  6 (shaded  region)  in  order  to  carry  out  the  algorithm. 


ktociel  of  the  Student; 

S jumped  past  the  motiv.ilmg  factorr.  to  the  travcis»il  algc*rithm  itself. 
C'oiiif>.if  r.oM  of  l5ox  E>  to  the  Topic  Knowledge  Representation  (Figure  6)  snows  that  S 
tni-.n'l  KkMititied  all  necessary  conditions.  Particular  differences  €«re  that  believes  it 
Sufficient  to  consider  only  sub-trees  of  the  root,  and  sufficient  to  only  consider  one 
link  fc>r  nodes  m the  sub- trees. 
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Lino  33 

Plntxnei". 

The  level  of  pros(»ntation  policy  is  becoming  increasingly  settled.  The  effect  is 
that  mid  lcvel  strategies  will  be  preferred,  e.g.,  suggesting  factors  to  the  student 
rather  than  asking  S to  list  them  or  restating  factors  already  given. 

There  is  a problem  applyir^g  "Counter-example:  insufficient  factor"  (Table  1,  Rule 
6),  since  there  are  two  cases  of  insufficient  factors:  printing  the  root  and  considering 
all  links  are  both  essential.  The  "dependent/indopendent  situation"  conflict  heuristic 
resolves  this  problem. 

For  ^ trees,  printing  the  root  wilt  improve  the  output  of  S's  algorithm  in  Box  8. 
Only  for  some  trees  does  the  second  factor  improve  results,  since  trees  similar  to 
Figure  won't  evoke  the  second  bug.  Since  considering  links  is  relevant  only  in 
situations  comprising  a sub-set  of  the  criticial  situations  for  root-printing,  the  latter  is 
the  independent  condition.  Thus,  root-printing  is  chosen  as  the  topic  for  Rule  6. 

"Counter-example:  insufficient  factor"  is  chosen  only  after  "Generalise: 

ii^suf ficient  factor"  and  "Ocivionstrate:  missing  factor"  (lable  1,  Rules  5 and  9 
respectively)  are  rejected.  S has  predicted  a value  on  "traversal".  This  is  a task;  its 
values  are  possil>le  algorithms  (only  one  of  which  is  under  consideration  here).  S has 
clc'sc nl;»ed  an  algorithn^,  though  not  quite  the  correct  one;  that  is,  S has  hypolhesiired 
the  factors  necess»iry  for  the  algorithm  to  work.  Therefore,  any  rule  dealing  with 
assertions  about  factors  is  likely  to  apply.  Rule  6 is  proterrod  over  Rule  5,  despite 
thc'ir  idemtical  conditions,  because  of  the  mid-level  presentation  policy  decision 
described  iibove.  Rule  9 is  quite  similar  to  Rule  6,  but  is  eliminated  by  the  "specificity" 
conf lie -re solution  heuristic. 

Carrying  out  "Counter-example:  insufficient  factor"  (the  winning  rule)  calls  for  a 
disconfii  ruing  value  on  the  dependent  variable.  Any  input  tree  suffices;  the  factors 
given  by  S fad  to  predict  the  root  appearing  in  the  output  sequence  for  the  traversal 
algorithm. 

Topic  Knotvltdire  l{ri)trsentaiion: 

Output  for  the  tree  of  Figure  Aa  can  be  obtained  by  executir>g  the  correct 
algorithm  (Figure  6)  with  the  tree  as  input.  Obtaining  information  about  processes  by 
siiviuliiting  them  m this  manner  has  become  an  issue  of  some  recent  interest  (see 
Brown,  Burton,  a»ul  Bell,  197A;  Brown  and  Burton,  J975;  Scragg,  1975). 

^todvl  of  the  Student: 

In  conjunction  with  Topic  Kno.vlcdge  R(?presentation,  the  Model  of  the  Student 
can  predict  the  student's  algorithm’s  output  by  executing  the  correct  algorithm,  but 
considering  only  the  components  marked  as  appearing  m the  student's  version. 
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I UU-.  :•;/! 

Mod^l  of  tha  Sludent: 

S has  now  correctly  described  the  output  for  the  algorithm  as  realized  in  Box  8. 


l inos  ^0-dl 

Mod^L  of  the  Student: 

Tfie  student  is  now  aware  of  the  way  in  which  the  proposed  algorithm’s  output 
was  incorrect,  and  that  the  missing  necessary  factor  was  printing  the  root  between 
consiflcMing  left  and  right  links.  Thus,  S now  Knows  one  previously  neglected 
necessary  factor  of  the  algorithir»,  but  others  reiriain:  S still  thinks  it  sufficient  to 
(‘xamino  ir.dt  links  only  when  following  the  root’s  left-hand  sub-tree,  and  right  links 
only  wlien  in  the  right-hand  sub-tree. 


Linos 

V’o/jir  Knou-lrtl fte  /irprrxnntntion: 

Traversing  a binary  sort  tree  correctly  requires  --  *or  all  nodes  --  following 
their  left  link,  then  printing  them,  then  following  their  right  link  (Figure  6). 

Mod^l  of  the  Student: 

S had  believed  that  1)  only  left  links  need  be  followed  for  nodes  in  the  left  stb- 
tree,  and  '/)  only  right  links  need  be  lollowed  for  nodes  in  the  right  sub-tree. 

In  those  linos  belief  (i)  is  clearly  being  modified.  It  is  unclear  whether 

belief  (2)  has  also  changed;  the  conservatcve  approach  is  to  assume  not. 


I I Mr  ‘11  1 

riunnc.r: 

In  Lino*.  S modified  only  one  of  two  incorrect  assumptions.  This  drops 

the  presentation  level. 

As  at  Line  33,  the  applicable  rules  are  5,  6,  and  9.  S «s  again  ignoring  a 
nf‘Cossary  factors  (handling  left  links  of  nodes  m a tree’s  right-hand  side).  Arguments 
similar  to  those  for  I mo  33  apply  to  rule -selection  hero,  except  that  the  •owered 
pre$ent«*ition  level  causes  selection  of  the  simpler  "Generalize:  insufficient  factor"  rule 
( I ablp  1 , Pule  b'. 

S's  hypotho'.is  appoars  to  b#  that  left  sub  free  nodes  require  following  both 
links,  while  rigtit  nodes  require  following  only  right  links  S «s  asked  if  this  is  correct. 
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Model  of  the  Sltidetil: 

Model  of  the  Stucienrs  anaiy««is  of  lines  43-44  is  now  confirmed.  S understands 
the  correct  rule  for  one  case,  but  has  not  transferred  that  understanding  to  the  other. 


L.in(>  43 

Model  of  the  Slndeul: 

S finally  n?cogni7es  the  necessity  of  following  both  links  of  all  nodes.  The 
current  situation,  then,  is  this:  S has  stated  an  algorithm  (Box  9)  which  fails  in  certain 
cases,  awareness  of  those  cases  exists,  but  appropriate  fixes  aren’t  yet  formulated. 


1 me*-  hO  [14 

Topic  Kuouiltd iie  /Representation: 

The  rK*w  algonihm  (Box  10)  is  functionally  equivalerit  to  the  correct  algorithm  of 
f igur  i?  6,  sin<(‘  the  FCILOWL  and  FOLLOV^R  procedures  are  identical  both  to  each  other 
and  to  the  last  three  lines  of  OUTPUT. 

Model  of  the  Student: 

Inc  Topic  Knowledge  Representation  analysis  above  suggests  two  things:  S 
distingu.sh.es  between  left-lmK  and  right-lmr.  nodes  (which  is  unnecessary),  and  5 
distinguishes  between  the  root  and  nodes  below  it  (also  unnecessary). 


i in<* 

/*/nnn'*r: 

"Oneralije:  unnecessary  factor**  (labic  I,  Rule  7)  is  selected  Over  the  one 
a!ti‘i  nativc>,  **Cour>ter  cvample:  unnecessary  factor”  (Table  I,  Rule  8).  Both  rules  can 
be  conditioned  solely  upon  a sufficient  factor  being  thought  necessary. 

Rule  8,  as  the  higher  present«MiOn  level  rule,  woulci  be  preferred.  Unforturiately, 
it  calls  for  finding  a case  with  a “wrong**  value  on  the  factor  and  t.he  correct  value  on 
the  (jependent  variable.  Since  the  stucient's  version  of  the  algorithm  is  functionally 
equivalent  to  the  tutor's,  no  such  value  can  exist.  ® Thus,  the  "blocked -action” 
conflict -resolution  heuristic  forces  rejection  of  Rule  8 m favor  of  Rule  7. 

N«**t  to  be*  rc'soivrci  is  which  unnecessary  factor  Rule  7 will  act  on.  This  is 
doc  idl'd  by  the  conflict-resolution  heurist*:  “dependent/independent  situations”. 

Reducing  the  last  three  lines  of  OUTPUT  in  Box  10  to  a single  line,  "FOILOW(ROOT)", 
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II M ‘I  i.'fori 
Icfl/ri^ht 
ieft/ni»ht 


on  n.,v.ns  rndncccl  FOU.OWL  ,-.nd  FOUOWR  to  the  Mncle  procedure  FOLLOW 
i>.  emn,n..t,ng  (ho  rool/cub-lree  distinction  depends  on  first  eliminating  the 
sub-tree  distinction.  This  is  done  in  Line  55  simply  by  asking  S if  the 
cJiDtinction  ffi  necessary. 


Lii’ios  56-b7 


Modtl  of  th^  StaHvni: 


S now  io(OBni.-os  that  the  lett/nuht  distinction  is  not  necessary.  This  eliminates 
one  misbelief  but  also  confirms  another’s  existence:  S distinguishes  th^  root  froo: 

tl  o'diiiopue  the  conclusion  of 

dialogue,  are  more  Or  less  a repetition  of  the  pattern  just  presented. 
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PART  III  --  SUMMARY 

Whal  h.'ic.  jUf.t  been  presented  m Pari  II  constitutes  a fairly  detailed  analysis  of  a 
tutorial  dialof.uc.  When  operating  at  that  level  of  detail  for  such  an  extended 
ciiscciSGion,  the  big  picture  of  the  overall  system  is  likely  to  be  lost.  Let  us  step  back 
and  revu?w  this  picture;  we  will  first  summarize  the  dialogue  analysis,  then  step  back 
furlhi  r lo  review  the  r.ysteivi,  then  step  back  still  further  to  consider  what  is  involved 
in  actually  implementing  an  Intelligent  Educational  Dialogue  System. 

One  of  the  major  things  illui.trated  by  the  analysis  is  that,  given  only  a simple 
sc't  of  policies  aiKl  conflict-resolution  heuristics,  it  is  possible  to  make  reasonable 
selections  of  tt^achl^g  rules.  In  only  three  of  the  tutor’s  eight  turns  was  there  a 
unique  rule  which  could  be  applied.  Even  in  one  of  those  cases,  a decision  must  be 
made  between  two  alternative  forms  of  the  rule  (Lme  i).  Nevertheless,  it  always 
scervis  to  be  possible  to  find  a unique  rule. 

The  selection  process,  rather  than  being  prc-dctcrmined,  is  actually  fairly 
dexil'ilc.  F'or  exampln,  the  same  rules  apply  at  Line  33  and  at  Lme  45:  "Generalize: 

• nsiifficiont  factor",  "Counter -example:  .r^sufficieni  factor",  and  "Demunstrate:  missing 
lactor"  < f able  I,  Rules  5,  6.  and  9,  respectively).  At  Lme  33,  Rule  6 was  selected,  but 
at  Line  45  the  system,  rather  than  scteclmg  Rule  6 again,  reflected  a change  in  the 
.ov'(.'l  of  presentation  by  selecting  Ri  le  5. 

Often,  a number  of  decisions  have  to  be  made  before  the  action  to  be  taken  is 
finally  determ  ned  On  Lines  33  and  55,  we  saw  cases  where,  after  a rule  had  been 
selected,  it  was  necessary  to  i hooise  between  alternative  situations  to  which  it  could  be 
applied 

We  also  saw  tfie  system  make  use  of  a great  deal  of  knowledge  about  the  topic 
matter.  If^  Lines  8 thro  igh  25,  various  knowledge  about  consicJerations  of  a knov'n 
put  1 of  tfio  bii'iary  sort  algoiithrri  winch  motivate  the  design  of  the  unknown  part  is 
u‘.ed  in  leading  tfie  student  to  develop  hypoU'eses  about  wiiat  the  unknown  par)  (the 
trave  rsal  algorithm)  must  look  like.  In  Lines  33  through  56-57,  knowledge  about  the 
correct  form  of  the  traversal  algorithm  is  used  to  detect  and  diagnose  problems  with 
tfie  student’s  version, 

V.'n.it  c.m  l>e  concluded  al>out  the  structure  of  the  system  from  examining  tf’u 
cfiaiogue  anal /MS?  Clearly,  the  Key  part  of  the  system  is  the  Planner;  more 
paf  tie  ul.irly,  within  the  Planner,  key  roles  arc  played  by  trni'hing  whicfi 

determiive  actions  to  take  m particular  situations,  by  cnHflui-rviolution  l%ruri%tic%, 
whicli  choM<  among  the  teaching  rules  when  more  than  one  applies,  and  by 
•♦•Min/f.  which  specifics  general  constraints  that  restrict  the  range  of  possible  actions 

i Me  Pl.*nf\ei,  .illfuuig.h  *he  UMViiig  of  tfie  systeru,  is  -.till  neavily  clependerd 

cm  •.<•1  vices  rendered  hy  iMe  other  conponents  of  tfu'  systc-m  This  paper  fias 
priMi.irily  (cmierrird  itself  Only  with  ir'.e  Model  of  tfic  Stuilfnt  and  Topic  Knowledge 
f'c'pr esent.ition.  Information  abOut  the  nature  of  the  student  and  the  nature  of  the 
Mil>je<  t matter  is  essential  to  a nnowledgcable,  fle»  .vie  teach.ng  system.  These  two 
compcirvents  are,  m turr\,  efrpendont  on  services  rendered  by  each  olncr  (and  by  other 
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ionip(»iu?nt^  tu)t  it^  thio  paper).  For  example,  the  Model  of  the  Student  makes 

uM.‘  of  lopiL  Knov^lo(l{'c  R<‘prc5.entation  as  a basis  against  which  to  measure  the 
t ucJent Knov/ledge. 

I!.  noedtHl  to  iictiially  implnmrnt  an  Intelligent  Educational  Dialogue  System? 
Miicti  of  the  woi  I.  woiilii  ccmi.e.t  of  fleshing  out  the  outline  sketched  in  this  p«aper  The 
/Output  ll.indli.r,  for  c^xainple,  requires  imnlementtition  of  a parser  and  text 
lU'i  ator:  miinerous  schemes  for  both  tasks  already  exist  which  are  capable  of 
hciiuilinj;  an  adequate  i ange  of  natural  language.  Further,  improved  versions  of  the  I/O 

l l.  nulls  r wouUf  be  relatively  easy  to  insert  into  the  system  at  a later  date,  since  each 
component  of  the  system  is  simply  a “black  box“  from  the  viewpoint  of  the  other 
components. 

A number  of  other  instances  where  fleshing  Out  would  be  required  appear  in  the 
comf'Oiu'nts  highlighted  m this  paper.  Choices  between  alternative  teaching  rules  arc 
fiec|ui'ntly  m.ide  On  thc‘  basis  of  differences  in  the  level  of  sophistication  they  demand 
of  .1  student;  the  tearlur.g  rules  must  therefore  be  systematically  tagged  wnth 
inioi  iri.dioii  .ibout  tfieii  levr'l.  The  condition  pai  ls  of  teaching  i ules  are  to  be 
iiiipii  Inf  nted  Ihroujdi  the  mechanism  of  Knowledge  State  Conditions,  which  wa*. 
pi  e*  ••ntr  ii  m P.ii  t l‘s  discussion  of  the  Model  of  the  Student.  Thus,  KSCs  (which  consist 
of  an  tnfoi  in.dion  (ollrcloi  . an  anaiysis  section,  and  a wrighter)  need  to  be  specified 
foi  e.i(h  toachin-;  rule  condihon  pait.  In  addition,  general  support  procedures  for 
• ; It  yinj'.,  invcKing,  and  running  KS(X  would  need  to  be  implemented;  processing  of 
till-  KSc's,  it  slioulii  Ifi-  I rmf'inl/iM  i.Hl,  is  .»  shaii'ct  responsibility  of  Planner  ancf  Model  of 
tin  '/iliidunt,  I iiuilly,  a thoiough  database  o!  knowled^,e  about  the  topic  matter  needs 
to  be  built  into  lo;»tr  Knowledge  Representation;  this  paper  has  si’iown  some,  but  not 
nearly  all,  of  tin*  KtUiwU  dge  that  needs  to  be  represented. 

There  aie,  m addition,  thiee  major  technical  issues  which  must  be  dealt  with. 
Ti-.ey  l^ave  not  been  addressed  in  this  naper,  as  they  arc  well  beyond  its  scope,  but 
wori*  on  i‘a(h  is  essential  to  final  iiviplementation  of  a system.  The  first  issue  lies  m 
the  a|)|»li(  alion  of  lire  conflict-resolution  heuristics.  A systematic  procedure  needs  to 
l>e  •.perified  for  deter mir^mg  which  heuristic  will  be  used  m clK>Osing  between 
«iltiM  n.itiv('*.  .it  a giv<‘M  point 

I he  •..«'<  ond  .imf  !hi»if  r.Mie*.  are,  respectively,  tliose  of  rnntrol  Ururmrr  ancf 
iiifei  t Tiu'*.e  aie  closely  related  The  compoiu^nts  described 

11^  till'.  p.ipiM  art*  pruci'sses  s))eciaii/ed  tor  particular  tasKs;  each  controls  many  sub- 
pi  oh“.m'*.  A»  (niiijHmt*nt  doe*,  its  processing  wh.enevrr  necessary.  Thus  two 
< t»mptiiM*nt*.  m.»v  '.onu  tiinr*  c»p»*r.«te  m paiallcl,  oUvr  times  s€*ri.itly,  arnl  still  other 

lm. f  to  foiitiiw.  Ihiu  *uggr*.ts  .1  p.iiticular  iontrol  structure:  a distr-buted 

pi  i»*  iv«-|wiirl.  Ii>  *.»i(h  a turn  luer art  l•.:f  al  control  stimlure,  Urere  is  rvo 

t niitt  oliif^;  pirn***.  !»uf  fathfi  i (»  ttp«M  atiun  iMd-'Ocnco  piocf.sev. 

Wf  ri»  i-tl  to  .t'.k  hua^  .tiih  ..  iontroi  •.true!  would  be  implemented,  aruJ  how 
pMUt  '.t*  oper.ttmg  tin-,  stiuctiiii*  wOuld  COmn.jnicate  willr  each  OltM*r.  T lie 
.tn-  wi  •*.,  it  ‘.retu*.  Ifi.flv,  lie  m tire  ciweetiOn  Of  viewing  each  comportent  as  a processor 
wifh  its  own  “operating  *.y*  tern"  Comrt'unicatiOns  between  comporvnts  wOuld  take 
plate  by  riir*%t.age  passing  between  tlicse  “operating  systems*  which  would  cJetermine 
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the  pnonlies  wilh  which  messages  would  receive  attention  and  would  take  care  of 
I ijptmg  on-going  processing  when  necessary.  A great  deal  of  the  future  work  to 
bo  done  on  implementing  an  Intelligent  Educational  Dialogue  System  will  consist  of 
rpocifymg  these  "operating  systems"  and  defining  the  protocols  for  passing  messages 
between  them. 
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FOOTNOTES 


/h'l(i\nu)l^(l f’owrnta:  Gr.iteful  .icknowled[>ement  is  made  to  Donald  A.  Normati, 

wlio  provi(i(?d  comments  and  suggestions  at  every  stop  of  this  study.  This  research 
was  supported  by  the  AdvaiKc-.'cl  Research  Projects  Agency  and  the  Office  of  Naval 
f?(n.(Mrrh  of  the  Department  of  Defense,  and  was  monitored  by  ONR  under  Contract  No. 
NOOO I ^-7G-C  0G23.  Final  cli  afts  of  this  report  were  prepared  with  the  use  of 
facilities  providcfd  by  the  departments  of  Psychology  and  Computer  Science  at 
Car nc‘gie- Mellon  University. 


^ "Build  tree"  is  a portion  of  the  binary  tree  sort  algorithm. 

^ In  gcner.il  policy  terms,  the  Socratic  method  considered 
in  Part  I v/o  is  a teaching  strategy  with  presentation 
c hat  acteri,red  by:  1)  contingency-driven  sequence,  2)  question- 
oriented  manner,  and  3)  tutor-driven  initiative.  (General  and 
specific  issues  of  level  will  be  discussed  later  on.)  Analysis  of 
hov/  gener«al  policy  decisions  are  made  rAust  await  comparison 
of  alloi  native'  teaching  strategies.  This  requires  representation 
of  otlufr  stratc'gies  using  the  same  production  paradigm,  long- 
I angc  goal  of  work  in  progress  by  the  author  at  Carnegie-Mellon 
IJnivc'ir.ily. 


^ This  method  (Knuth,  1973)  orders  a list  of  items  by  first 
building  a binary  tree*  data  representation  for  the  list,  then  using 
a recurj.ivc*  tree  traversal  rule  to  select  items  from  the  tree  in 
their  correct  order.  The  Key  to  the  algorithm  is  that  the  tree  can 
Ih‘  built  to  have  a property  that  makes  sucvcssful  traversal  quite 

Mivtplr. 


^ Subji'ct*-  wore  inostly  sophomores  at  the  University  of 
California,  S.m  Du’go,  with  less  than  a year  ot  programming 
expc'nencc  ir»  Al.C'iOl.  In  'he  m.ijOrity  of  the  tutorials,  students 
wt'ie  .I'.kod  to  hand  •.im'il.ito  .ilgonthms,  but  were  not  asked  to 
state*  thorn  as  (orupulei  progCiims. 
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^ In  AI  methodology,  this  is  a first  test  for  a proposed 
system.  It  is  a minimal  test;  a system’s  ability  to  explain 
intelligent  behavior  is  a necessary  --  but  not  sufficient 
condition  for  the  ability  to  generate  it. 

^ The  problem  is  ii*k  applying  Collins’  rules  to  the  Topic 
Knowledge  Representation  for  algorithms;  a distinction  has  been 
created  between  unparsimonious  (as  opposed  to  unnecessary) 
factors  which  the  rules  are  not  equipped  to  handle. 
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Figure  ? 

The  Topic  Knowledge  Representation  node  network 
for  the  following  statements: 

U)  Iherc  are  two  ways  to  SORT,  the  STRAIGHT 
SELECTION  SORT  and  the  BINARY  TREE  SORT. 

(2)  The  BINARY  TREE  SORT  is  a fast  algorithm. 

<3)  The  BINARY  TREE  SORT  consists  of  two  tasks: 
first  build  a tree,  then  traverse  it. 

(4)  When  sorting  requires  speed,  use  BINARY  TREE 
SORT  because  it*s  fast. 


4a.  A very  simple  binary  sort  tree. 


1.  3 


4b.  The  output  sequence  obtained  by 
applying  the  program  of  Box  8 to 
the  free. 


1.  2,  3 


4c.  The  correct  output  sequence  for 
the  tree. 


Figure  4 

A simple  example  binary  sort  tree,  used  to  demonstrate 
a flaw  in  the  student's  statement  of  the  algorithm.  The 
flaw  can  be  seen  by  comparing  the  student  results  to 
the  correct  results. 


5a.  A tree  presentif^  problems  for  the 
program  of  Box  9. 


1,  3.  18.  24 


5b.  Output  sequence  for  the  program  of 
Box  9 on  The  above  tree.  (Note  that 
the  number  “Q"  was  missed.) 


5c.  Another  tree  presenting  problems  for 
the  program  of  Box  9. 


Figure  5 

Demonstrations  of  problems  in  the  student's 
restated  version  of  the  algorithm. 
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Table  2 

Structures  needed  for  applying  Collins  rules 
and  their  dcntifyjrg  cba^'actenstics  in  TKR. 

(for  ALG0R:TH»!3) 


I2ble  3 

Structures  needed  for  applying  Collins  rules, 
and  the:r  identifying  characteristics  in  TKR. 
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Other  Government 

1 Dr . Vern  Ur  ry 

Personnel  R4D  Center 
US  Civil  Service  Commission 
1900  E Street  NW 
Washington  DC  2B415 

1 Dr . Anorew  R.  Molnar 

Science  Eoucation  Dev.  i Res. 
National  Science  Founoation 
Washington  DC  20550 

1 Dr.  Marsnall  S.  Smith 
Assoc.  Director 
Mt/CPEPA 

National  Institute  of  Eoucation 
Washington  DC  ?f209 

1 Dr.  Joseph  L.  Young,  Director 
Memory  s Cognitive  Processes 
National  Science  Foundation 
Washington  DC  20550 

1 Or . James  M.  Ferstl 

Employee  Development  Training 
Technologist 
Bureau  of  Training 
US  Civil  Service  Commission 
Washington  DC  20415 

1 Will iam  J . McLaur in 
Rm.  101 

Internal  Revenue  Service 
2221  Jefferson  Davis  Hwy. 
Arlington  VA  22202 


Miscellaneous 

1 Dr.  John  R.  Anderson 
Dept,  of  Psychology 
Yale  University 
New  Haven  CT  06520 

1 Dr.  Scarvia  B.  Anderson 
Educational  Testing  Service 
Suite  1040 

3445  Peachtree  Rd . NE 
Atlanta  GA  30326 

1 Prof.  Earl  A.  Alluisi 
Code  287 

Dept,  of  Psychology 
Old  Dominion  University 
Norfolk  VA  23508 

1 Dr.  Daniel  Alpert 

Computer-Based  Education 
Research  Laboratory 
University  of  Illinois 
Urbana  IL  61801 

1 Ms.  Carole  A.  Bagley 
Applications  Analyst 
Minnesota  Educational 
Computing  Consortium 
1925  Sather  Ave. 

Lauderdale.  MN  55113 

1 Dr  . John  Brackett 
SofTech 

461  Totten  Pona  Kd . 

Waltham  MA  02154 

1 Dr.  Robert  K.  Branson 
lA  Tully  Bldg. 

Florida  State  University 
Tallahassee  FL  J2306 

1 Ur.  John  Seeley  Brown 

Bolt  Betanex  k Newman,  Inc. 

SB  Moulton  St . 

Cambridge  MA  02138 

1 Ur.  Victor  Bunderson 

Institute  tor  Corsputer  Uses  in  Ed 
1S5  EDLC 

Brigham  Young  University 
Provo  UT  84601 

1 Ur.  Ronald  P.  Carver 
;>chool  ot  td  lie  ,1  iron 
University  ol  Mrssoull 
5100  Rockhill  Kd. 

Kansas  City  MO  64110 

1 eentury  Research  Corp. 

4111  Lee  Hwy . 

Ar  1 rngion  VA  <220  7 

1 Jack  1 yn  Case  1 1 1 

ERIC  C lear  inghouse  on 
Intorrsation  Resources 
M aniord  Univer  sity 
Sihool  of  tdUi.ation  .Sv  RUT 
.M  am  or  J CA  94  105 

1 Dr  . Kenneth  E.  Clar  k 

lollegr  ol  Arts  6 sciences 
Univer srty  ot  Rochcat c: 

River  campus  Station 
Rochrsrn  ntt  1462' 

1 Ui  . <4l  lai:  M.  coll  ins 

Bolt  Hrranek  k N-man  in.. 

50  Moulton  .St  . 
c amt  1 tdge  MA  ».  1 >e 

1 Lr  . Jol.n  J . efi  1 in» 

Isses  Corp. 
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1 Dr.  Donald  Dansereau 
Dept,  of  Psychology 
Texas  Christian  University 
Fort  Worth  TX  76129 

1 Dr . Ruth  Day 

Dept,  of  Psychology 
Yale  University 
2 Hillhouse  Ave. 

New  Haven  CT  06520 

1 ERIC  Facil ity/Acquisitions 
4833  Rugby  Ave. 

Bethesda  MD  20014 

1 Dr.  John  Eschenbrenner 

McDonnel  Douglas  Astron.  Co.  East 

PO  Box  30204 

St.  Louis  MO  80230 

1 Major  I.N.  Evonic 

Canadian  Forces  Perso.nnel 
Applied  Research  Unit 
1107  Avenue  Rd. 

Toronto  Ontario  CANADA 

1 Dr.  Victor  Fields 
Dept,  of  Psychology 
Montgomery  College 
Rockville  MD  20850 

1 Dr.  Edwin  A.  Flwishman 

Advanced  Research  Resources  Org. 
8555  Sixteenth  St. 

Silver  Spring  MD  20910 

1 Dr.  Larry  Francis 

University  of  Illinois 
Computer-Based  Educ.  Research  Lab 
Champaign  IL  61801 

1 Dr.  Frederick  f.  Fricx 
MIT  Lincoln  Laboratory 
Rn.  D 268 
PO  Box  73 

Lexington  MA  02173 

1 Dr.  John  R.  Prederiksen 
Bolt  Beranek  k Netiman  Inc. 

50  Moulton  St . 

Cambridge  MA  02138 

1 Dr.  Vernon  S.  Gerlach 
College  of  Education 
146  Payne  Bldg.  B 
Arizona  State  University 
Tempe  AZ  85281 

1 Dr.  Robert  Glaser,  Co-Director 
University  of  Pittsburgh 
3939  O'Hara  St. 

Pittsburgh  PA  15213 

1 Dr.  H.D.  Havron 

Human  Sciences  Research  Inc. 

7710  Ola  Spring  House  Rd. 
rest  Gate  Industrial  Park 
McLean  VA  22101 

1 Dr . Duncan  Hansen 
school  ot  Education 
Memphis  State  University 
Memphis  TN  38118 

1 Human  Resources  Research  Org. 

400  Plaza  Bldg. 

Pace  Blvd.  at  Fatrftsld  Drive 
Pensacola  FL  32505 

1 HumRRO/Nettern  Division 
27857  Berwick  Di ive 
Caimel  CA  93921 
Attni  Libisiy 

1 HumRRO/Columbua  Oft  tee 

Suite  23,  2601  Ciuas  Country  Di  . 
Columbus  CA  31906 

1 HumRRO/Ft.  Knox  Office 
PO  Box  293 
Fort  Knox  K¥  4B121 

1 Ur.  Lawrence  B.  Johnson 

Lawrence  Johnson  6 Assoc.  Inc. 
Suite  512 
2011  S Street  NW 
Washington  UC  2000« 

1 . Arnold  F.  Rsnsitck 

Honeywell  Inc. 

2600  Ridgeway  Pkwy. 

Minneapolis  NN  55413 

1 Ur  . Roger  A.  Ksutmsn 
20 i Uodd  Hall 
Fluilds  State  U<ilveiatty 
Tallahassee  FL  12106 

1 Ut . Steven  W.  Reele 
tvpt  . ot  Psychology 
University  ot  Oregon 
tugene  uR  9 f 40  I 

1 Di  . Uav  Id  K1  ah  I 

Uepi . ol  Psychology 
i ai  nrg re ;Mr i I on  university 
lilt  slioi  gi.  lA  15.11 

1 Ur.  Rot'eil  R.  Ma..kir 

Kuean  Fa.  lots  Reseat  cl.  Inc. 
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1 Dr . Leo  Munday 

Houghton  Mifflin  Co. 

PO  Box  1970 
Iowa  City  lA  52240 

1 Mr.  Thomas  C.  O'Sullivan 
TRAC 

1220  Sunset  Plaza  Drive 
Los  Angeles  CA  90069 

1 Mr.  A.J.  Pesch,  {resident 
Eclectech  Assoc.  Inc. 

PO  Box  178 

N.  Stonington  CT  06359 

1 Mr . Luigi  Petrullo 
2431  N.  Edgewood  St. 

Arlington  VA  22207 

1 Dr.  Steven  M.  Pine 
N 660  Elliott  Hall 
University  of  Minnesota 
75  Eiist  River  Rd. 

Minneapolis  NN  55455 

1 Dr.  Kenneth  A.  Polycyn 

PCR  Information  Sciences  Co. 
Communication  Satellite  Applications 
7600  Old  Spr inghouse  Rd. 

McLean  VA  22101 

1 Dr.  Diane  M.  Ramsey-Klee 

ROK  Research  k System  Design 
3947  Ridgemont  Drive 
Malibu  CA  90265 

1 R.Dir.  M.  Rauch 
P II  4 

Bundesmir.ister ium  der  Verteidigung 
Postfact  161 
53  Bonn  1,  GERMANY 

1 Dr.  Joseph  W.  Rigney 
University  of  So.  Calif, 

Behavioral  Technology  Laboratories 
3717  South  Grand 
Los  Angeles  CA  90007 

1 Dr.  Andrew  M.  Rose 

American  Institutes  for  Resescch 
1055  Thomas  Jefferson  St.  NW 
Washington  DC  20007 

1 Dr.  Leonard  L.  Rosenbaum,  Chairman 
Dept,  of  Psychology 
Montgomery  College 
Rockville  MD  20850 

1 Mr.  Charles  R.  Rupp 

Advanced  W/C  Development  Kng. 

General  Electric  Co. 

101  Plastics  Ave. 

Pittsfield  NA  01211 

1 Dr.  Robert  J,  Seidel 

Instructional  Technology  Group 
HumRRO 

300  N.  Washington  St. 

Alexandria  VA  22314 

1 Dr . Richard  Snow 
Stanford  University 
School  of  Education 
Stanford  CA  94305 

1 Dr . Peiais  Sturgis 
Dept,  of  Psychology 
California  State  University 
thico  CA  95926 

1 Ml.  Walt  W.  Tor  now 
Contiol  Data  Coep. 

Coipoiate  Personnel  Reaeaich 
PC  Box  • HONB6B 
Minneapolis  MN  55441 

1 Di . K.w,  Unesphet 

Intoimstion  Sciences  Institute 
4676  Admiralty  Way 
Manna  Del  hay  CA  9B291 

1 Di . Benton  J.  Underwood 
Uept . ot  Psychology 
Hoithwestrin  University 
Evanston  IL  602B1 

1 Ui  . Call  P..  Vast 

Battellr  Memorial  Institutm 
Washington  UpetatlOhS 

2I3B  M Stiaet  NW 

Washington  Ot  2BB16 

1 Ul . David  J . Welaa 
Dept . ot  Psychology 
N66B  Elliott  Hall 
Univeiaity  ot  Hinnasota 
Htnneapolia  HH  55455 

1 Ul . Keith  Weacoult 
Uvpt . ol  Psychology 
Stantord  Univeiaity 
Stantoid  CA  94 115 

1 Or.  Claire  E.  Weinstein 

Educst tonal  Psychology  Dept. 

Untvei  atty  ol  Teaat 
Auat in  TX  7s 

I Ui  . Amt  a west 

Denver  Heaeaiih  Institute 
I'nivrteiiy  c-f  Denver 
Uenvei  ci>  SB2B1 
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